In recent years, many studies have been reported with regard to the super-resolution processing which estimates one high-resolution image from multiple low-resolution images having the displacement (see Non-Patent Document 1). Various methods of the super-resolution processing, for example, the ML (Maximum-Likelihood) method disclosed in Non-Patent Document 2, the MAP (Maximum A Posterior) method disclosed in Non-Patent Document 3, and the POCS (Projection Onto Convex Sets) method disclosed in Non-Patent Document 4 have been proposed.
The ML method is a method which defines an evaluation function as square error between the pixel value of a low-resolution image estimated from a high-resolution image and the actually observed pixel value, and obtains a high-resolution image by minimizing the evaluation function as an estimated image. In other words, the ML method is a super-resolution processing method based on the principle of maximum likelihood estimation.
The MAP method is a method which estimates the high-resolution image by minimizing the evaluation function which added probability information of the high-resolution image to square error. In other words, the MAP method is a super-resolution processing method which uses certain prior information regarding the high-resolution image to estimate the high-resolution image as an optimization problem that maximizes posterior probability.
The POCS method is a super-resolution processing method which generates simultaneous equations regarding the pixel values of the low-resolution image and the high-resolution image, and obtains a high-resolution image by solving the simultaneous equations successively.
All of the above-described super-resolution processing methods have the common features of presupposing a high-resolution image and estimating its pixel value for each pixel of all low-resolution images based on point-spread function (PSF) obtained from camera model from the presupposed high-resolution image so that these methods can search for a high-resolution image by minimizing the difference between the estimated value and the observed pixel value (the observed value). Therefore, these super-resolution processing methods are called reconstruction-based super-resolution processing methods.
One of the common features of the reconstruction-based super-resolution processing method is having a very high dimensional problem with an unknown number of the pixels of the high-resolution image, and another feature is the necessity to estimate low-resolution image from the high-resolution image for all pixels of multiple low-resolution images.
In the reconstruction-based super-resolution processing method, since the dimension of the unknown number of the pixels of the high-resolution image is very high, it is unrealistic to analytically derive the high-resolution image and thus the high-resolution image is estimated by iterative calculations. In addition, the iterative calculations need to estimate all pixels of the low-resolution images for one cycle. Therefore it is well known that there is a large calculation cost problem. That is to say, since the calculation cost of the reconstruction-based super-resolution processing is very large, a main problem of the existing super-resolution processing methods is to reduce the large calculation cost.
Moreover, the super-resolution processing defines square error between the estimated value and the observed value as the evaluation function of the estimated error, and estimates the high-resolution image as a result of the optimization calculation. Therefore, the evaluation function of square error and the derivative value of the evaluation function need to be calculated for the optimization calculation.
As described above, in the existing super-resolution processing methods, in order to calculate the evaluation function of square error and the derivative value of the evaluation function, it is necessary to calculate the estimated values corresponding to the observed values of all pixels of multiple low-resolution images. Therefore, it is necessary to estimate the total pixels time of multiple low-resolution images. Although the estimating calculation has been formulated as a convolution operation with the point-spread function (PSF) corresponding to the transfer function obtained from the camera model, it is necessary to perform the convolution operation for all pixels of multiple low-resolution images. Therefore, the total pixel number of multiple low-resolution images used in the super-resolution processing generally becomes 8,000 to 800,000, resulting in a very large calculation cost needed to execute the estimation.
The present invention has been developed in view of the above-described circumstances, and an object of the present invention is to provide a fast method of super-resolution processing which realizes speedup of the super-resolution processing by reducing the number of times of the convolution operations that is the number of times of estimation.